Scalable Subspace Snooping

Snooping tag bandwidth is one of the resources that limits the number of processors that can participate in a cache-coherent snooping system. In this paper, we evaluate a type of coherence protocol called subspace snooping, which decouples the snoop tag bandwidth from the address bus bandwidth. In subspace snooping, each processor snoops a set of logical channels, which are a subset of the total snoopable address busses in the system. Thus, each processor snoops a subset of the address space, reducing the number of tag matches required for a system of a given size. By dynamically assigning both processors and cache lines to channels, we support dynamic formation of subspaces, with the goal of having only sets of processors that share data snooping on each given channel. Subspace snooping aligns best with systems for which the address bus bandwidth greatly exceeds the snooping tag bandwidth. Snooping optical interconnects exhibit such characteristics, providing enormous transmission bandwidth, but which quickly become limited by snooping tag energy and bandwidth as the number of processors increases. Optical busses can be subdivided into logical channels using either wave-division or time-division multiplexing, making them good candidates for a subspace snooping implementation. We evaluate a range of subspace snooping protocols on six parallel scientific benchmarks, running on an execution-driven simulator. We show an average 50% reduction in total snoops for 64-processor systems with 8-32 channels, and that for regular applications, subspace snooping shows large performance improvements the ratio of processor bandwidth to snoop bandwidth grows large. This material is based upon work supported by the Defense Advanced Research Project Agency (DARPA) under Contract NBCH30390004.